With all the buzz around GNOME 3, I really wanted to try it out for real on my main laptop. It usually runs Debian Unstable but that s not enough in this case, GNOME 3 is not fully packaged yet and it s only in experimental for now. I asked Josselin Mouette (of the pkg-gnome team) when he expected it to be available and he could not really answer because there s lots of work left. Instead Roland Mas gently answered me Sooner if you help . First steps as a GNOME packager This is pretty common in free software and for once I followed the advice, I spent most of sunday helping out with GNOME 3 packaging. I have no prior experience with GNOME packaging but I m fairly proficient in Debian packaging in general so when I showed up on #debian-gnome (irc.debian.org) on sunday morning, Josselin quickly added me to the team on alioth.debian.org. Still being a pkg-gnome rookie, I started by reading the documentation on pkg-gnome.alioth.debian.org. This is enough to know where to find the code in the SVN repository, and how to do releases, but it doesn t contain much information about what you need to know to be a good GNOME packager. It would have been great to have some words on introspection and what it changes in terms of packaging for instance. Josselin suggested me to start with one of the modules that was not yet updated at all (most packages have a pre-release version usually 2.91 in experimental, but some are still at 2.30). Packages updated and problems encountered (You can skip this section if you re not into GNOME packaging) So I picked up totem. I quickly updated totem-pl-parser as a required build-dependency and made my first mistake by uploading it to unstable (it turns out it s not a problem for this specific package). Totem itself was more complicated even if some preliminary work was already in the subversion repository. It introduces a new library which required a new package and I spent a long time debugging why the package would not build in a minimalistic build environment. Indeed while the package was building fine in my experimental chroot, I took care to build my test packages like the auto-builders would do with sbuild (in sid environment + the required build-dependencies from experimental) and there it was failing. In fact it turns out pkg-config was failing because libquvi-dev was missing (and it was required by totem-pl-parser.pc) but this did not leave any error message in config.log. Next, I decided to take care of gnome-screensaver as it was not working for me (I could not unlock the screen once it was activated). When built in my experimental chroot, it was fine but when built in the minimalistic environment it was failing. Turns out /usr/lib/gnome-screensaver/gnome-screensaver-dialog was loading both libgtk2 and libgtk3 at the same time and was crashing. It s not linked against libgtk2 but it was linked against the unstable version of libgnomekbdui which is still using libgtk2. Bumping the build-dependency on libgnomekbd-dev fixed the problem. In the evening, I took care of mutter and gnome-shell, and did some preliminary work on gnome-menus. Help is still welcome There s still lots of work to do, you re welcome to do like me and join to help. Come on #debian-gnome on irc.debian.org, read the documentation and try to update a package (and ask questions when you don t know). Installation of GNOME 3 from Debian experimental You can also try GNOME 3 on your Debian machine, but at this point I would advise to do it only if you re ready to invest some time in understanding the remaining problems. It s difficult to cherry-pick just the required packages from experimental, I tried it and at the start I ended up with a bad user experience (important packages like gnome-themes-standard or gnome-icon-theme not installed/updated and similar issues). To help you out with this, here s a file that you can put in /etc/apt/preferences.d/gnome to allow APT to upgrade the most important GNOME 3 packages from experimental:

Package: gnome gnome-desktop-environment gnome-core alacarte brasero cheese ekiga empathy gdm3 gcalctool gconf-editor gnome-backgrounds gnome-bluetooth gnome-media gnome-netstatus-applet gnome-nettool gnome-system-monitor gnome-system-tools gnome-user-share baobab gnome-dictionary gnome-screenshot gnome-search-tool gnome-system-log gstreamer0.10-tools gucharmap gvfs-bin hamster-applet nautilus-sendto seahorse seahorse-plugins sound-juicer totem-plugins remmina vino gksu xdg-user-dirs-gtk gnome-shell gnome-panel dmz-cursor-theme eog epiphany-browser evince evolution evolution-data-server file-roller gedit gnome-about gnome-applets gnome-control-center gnome-disk-utility gnome-icon-theme gnome-keyring gnome-menus gnome-panel gnome-power-manager gnome-screensaver gnome-session gnome-settings-daemon gnome-terminal gnome-themes gnome-user-guide gvfs gvfs-backends metacity mutter nautilus policykit-1-gnome totem yelp gnome-themes-extras gnome-games libpam-gnome-keyring rhythmbox-plugins banshee rhythmbox-plugin-cdrecorder system-config-printer totem-mozilla epiphany-extensions gedit-plugins evolution-plugins evolution-exchange evolution-webcal gnome-codec-install transmission-gtk avahi-daemon tomboy network-manager-gnome gnome-games-extra-data gnome-office update-notifier shotwell liferea epiphany-browser-data empathy-common nautilus-sendto-empathy brasero-common
Pin: release experimental
Pin-Priority: 500
Package: *
Pin: release experimental
Pin-Priority: 150

The list might not be exhaustive and sometimes you will have to give supplementary hints to apt for the upgrade to succeed, but it s better than nothing. I hope you find this useful. I m enjoying my shiny new GNOME 3 desktop and it s off for a good start. My main complaint is that hamster-applet (time tracker) has not yet been integrated in the shell.

21 comments Liked this article? Click here. My blog is Flattr-enabled.

Inspired by World Backup Day, I decided to take a backup of my laptop. Thanks to using a free operating system I don't have to backup any of my software, just configuration and data files, which fit on a single DVD.

In order to avoid worrying too much about secure storage and disposal of these backups, I have decided to encrypt them using a standard encrypted loopback filesystem.

(Feel free to leave a comment if you can suggest an easier way of doing this.)

Cryptmount setupInstall cryptmount:

apt-get install cryptmount

and setup two encrypted mount points in /etc/cryptmount/cmtab:

backup  
  dev=/backup.dat
  dir=/backup
  fstype=ext2    fsoptions=defaults     cipher=aes

  keyfile=/backup.key
  keyhash=sha1    keycipher=des3
 

testbackup  
  dev=/cdrom/backup.dat
  dir=/backup
  fstype=ext2    fsoptions=defaults    cipher=aes

  keyfile=/cdrom/backup.key
  keyhash=sha1    keycipher=des3
 

Initialize the encrypted filesystemMake sure you have at least 4.3 GB of free disk space on / and then run:

mkdir /backup
dd if=/dev/zero of=/backup.dat bs=1M count=4096
cryptmount --generate-key 32 backup
cryptmount --prepare backup
mkfs.ext2 -m 0 /dev/mapper/backup
cryptmount --release backup

Burn the data to a DVDMount the newly created partition:

cryptmount backup

and then copy the files you want to /backup/ before unmounting that partition:

cryptmount -u backup

Finally, use your favourite DVD-burning program to burn these two files:

• /backup.dat
• /backup.key

Test your backupBefore deleting these two files, test the DVD you've just burned by mounting it:

mount /cdrom
cryptmount testbackup

and looking at a random sampling of the files contained in /backup.

Once you are satisfied that your backup is fine, umount the DVD:

cryptmount -u testbackup
umount /cdrom

and remove the temporary files:

rm /backup.dat /backup.key

Debian CD/DVD ISO images are huge files (4.7 GB for a DVD). It can take a long time to download if your connection with the server is not very fast. Jigdo can help you download them much more quickly if you have a high-speed connection to a normal Debian mirror. If your download rate from your usual Debian mirror is not better than the download rate from the closest Debian CD mirror, then you should download your images straight from the Debian CD mirror. Otherwise read on and learn how you can save lots of time. Jigdo is based on a simple idea: the CD images contain mainly thousands of .deb files (and source files too) that are already available on all Debian mirrors. So instead of downloading the ISO image in its entirety, you could reconstruct it with all the files that it contains, that you would download from your usual Debian mirror. Thus when you download a .jigdo file, you only download a little text file (compressed with gzip) that contains enough information to reconstruct the complete image:

• it contains a list of the individual files that are needed to rebuild the image;
• it indicates a mirror where those files can be downloaded (it s mainly a fallback URL in case our preferred mirror doesn t have them anymore);
• it contains an URL to a .template file which describes the layout of all files within the ISO image. It also contains the data which can t be downloaded from the mirror.

The tool that we will use to regenerate the ISO images is called jigdo-lite, it s available on Windows if you don t have any Debian system yet. Otherwise just install the jigdo-file Debian package. We just need the URL of a .jigdo file (get one from the Debian CD page) and we pass it to jidgo-lite on the command line. It asks for a path of Files to scan : if we have an older version of the same CD image, we can indicate a path where it is mounted and it will avoid redownloading the files that we already have, otherwise just press Enter. Then it asks for the Debian mirror to use and by default suggests the one used in /etc/apt/sources.list. Again pressing Enter is usually enough. You can even use --noask to avoid the prompts.

$ jigdo-lite http://cdimage.debian.org/cdimage/release/6.0.1/multi-arch/jigdo-dvd/debian-6.0.1-i386-amd64-source-DVD-1.jigdo
Jigsaw Download "lite"
Copyright (C) 2001-2005     jigdo@
Richard Atterer             atterer.net
Getting mirror information from /etc/apt/sources.list
Downloading .jigdo file
--2011-03-22 09:08:11--  http://[2001:6b0:e:2018::163]/cdimage/release/6.0.1/multi-arch/jigdo-dvd/debian-6.0.1-i386-amd64-source-DVD-1.jigdo
[...]
-----------------------------------------------------------------
Images offered by  http://cdimage.debian.org/cdimage/release/6.0.1/multi-arch/jigdo-dvd/debian-6.0.1-i386-amd64-source-DVD-1.jigdo':
  1: 'Debian GNU/Linux 6.0.1 "Squeeze" - Official Multi-architecture i386/amd64/source DVD #1 20110319-14:55 (20110319)' (debian-6.0.1-i386-amd64-source-DVD-1.iso)
Further information about  debian-6.0.1-i386-amd64-source-DVD-1.iso':
Generated on Sat, 19 Mar 2011 15:11:27 +0000
-----------------------------------------------------------------
If you already have a previous version of the CD you are
downloading, jigdo can re-use files on the old CD that are also
present in the new image, and you do not need to download them
again. Mount the old CD ROM and enter the path it is mounted under
(e.g.  /mnt/cdrom').
Alternatively, just press enter if you want to start downloading
the remaining files.
Files to scan: 
-----------------------------------------------------------------
The jigdo file refers to files stored on Debian mirrors. Please
choose a Debian mirror as follows: Either enter a complete URL
pointing to a mirror (in the form
 ftp://ftp.debian.org/debian/'), or enter any regular expression
for searching through the list of mirrors: Try a two-letter
country code such as  de', or a country name like  United
States', or a server name like  sunsite'.
Debian mirror [http://ftp.fr.debian.org/debian]: 
Downloading .template file
--2011-03-22 09:12:41--  http://cdimage.debian.org/cdimage/release/6.0.1/multi-arch/jigdo-dvd/debian-6.0.1-i386-amd64-source-DVD-1.template
[...]
Successfully created  debian-6.0.1-i386-amd64-source-DVD-1.iso'
-----------------------------------------------------------------
Finished!
The fact that you got this far is a strong indication that  debian-6.0.1-i386-amd64-source-DVD-1.iso'
was generated correctly. I will perform an additional, final check,
which you can interrupt safely with Ctrl-C if you do not want to wait.
OK: Checksums match, image is good!

And you re done, you have successfully downloaded an official Debian DVD with jigdo-lite. You can head over to the Debian CD page and find out the URL of the .jigdo files for the latest stable release. One last thing, Bluray images are only available with Jigdo because they are so huge that keeping complete ISO images would be a waste of disk resources of the CD images server (and its mirrors). Of course, if downloading ISO images and burning CD/DVD is not for you, you can always order some professionally printed CD/DVD. You will even help Debian by doing this.

If you want to read more articles like this one, click here to subscribe to my free newsletter. You can also follow me on Identi.ca, Twitter and Facebook.

6 comments Liked this article? Click here. My blog is Flattr-enabled.

Not for the first time, I ve seen the suggestion in the echo chamber that Mono packages should be moved from Debian into the non-free repository, which is not formally part of Debian. The reason, as it so often is, is patents specifically this time, the searing risk posed to Debian and its users that Mono s packaging does not (and technically could not without forking from upstream) provide base class libraries which implement only the content of ECMA-335 4th Edition. As I understand it, this implies three things about the suggestion/demand: firstly, that the individual in question believes that Mono end users are at risk from patent litigation from Microsoft Corp because Mono s implementation of Microsoft.NET beyond the content of ECMA 334/335 infringes on Microsoft patents; secondly, that the Microsoft Community Promise which promises not to assert legal claims over third party implementations of ECMA-335 4th Edition (and ECMA-334 4th Edition which defines the C# language) would render a pure ECMA-only runtime safe if it existed (which it does not); thirdly that without the protection offered by the Microsoft Community Promise, the source code licenses of Mono are irrelevant the patent risk renders the software non-Free. It appears, unfortunately, that the community of Free Software Advocates don t actually understand what Free Software actually IS. That explains an awful lot, but should surprise nobody. So here s a lesson on what, exactly, is being advocated for. The Free Software Foundation defines four Software Freedoms these are the minimum criteria required for something to be considered Free Software by the FSF:

• The freedom to run the program, for any purpose (freedom 0).
• The freedom to study how the program works, and change it to make it do what you wish (freedom 1). Access to the source code is a precondition for this.
• The freedom to redistribute copies so you can help your neighbor (freedom 2).
• The freedom to distribute copies of your modified versions to others (freedom 3). By doing this you can give the whole community a chance to benefit from your changes. Access to the source code is a precondition for this.

Other groups have their own variants on these, but those are really just clarifications on the FSF definition - for example, the Debian Free Software Guidelines mostly line up, but have some additional clauses such as clause 4 which allows software to be considered Free if source code may be redistributed without modifications, as long as patches may be shipped alongside. These four freedoms are offered to you by the software s copyright holders only, and apply regardless of their choice of license any Free license, from a lengthy legal tome like the MPL to the completely-Free WTFPL, will offer you these four basic freedoms as a minimum, and any additional clauses in their licenses cannot seek to restrict these freedoms. These four freedoms represent the beginning, and end, of whether a piece of software is Free or not. Software does not need to be developed in the open, in a community-responsive way, in order to qualify as Free projects such as Google s Android, which are developed under a throw a final release over the wall, bugs and all, and expect people to thank you for it model, are still free, even if contribution is difficult. Actually, on a related note, software does not need to solicit upstream contribution of any kind in order to qualify as Free as long as you personally can redistribute the code with changes, then that s enough. Software does not need to serve a fully or even partially legal purpose in order to qualify as Free the favoured tool for causing distributed denial of service attacks, Low Orbit Ion Cannon, is Free Software, even though realistically it serves no legal purpose. DeCSS, the code initially used to allow DVD media to play on Linux (by breaking the CSS encryption mechanism) is Free Software. Software does not need to be useful or tasteful in order to qualify as Free the Last Measure Operating System, a minimalist OS primarily designed to loudly display the famous shock site images from goatse and related, is Free Software. Even in somewhat less clear-cut cases of taste, your personal opinion of software has no bearing on whether it is Free Software, as long as the four freedoms are guaranteed by the author(s). Software does not need to have only Free dependencies to qualify as Free Software it is entirely permissible to write software which relies upon a non-Free framework or library, and release your code under a Free license. It is the downstream recipient s problem to provide the dependencies, including their choice to craft a Free replacement for any non-Free code you make use of. Debian has a special repository called contrib, where Free software which only works with non-Free data, lives for example, Free game engines which require the insertion of proprietary game data in order to operate. You could write Free addons for expensive proprietary software such as Matlab, and as long as your code is Free, your responsibilities are met. Software does not need to avoid patents software, algorithmic, or otherwise in order to qualify as Free. The Freetype font library was still entirely Free Software when Apple were slinging threats around regarding font hinting data. FFmpeg, the Swiss Army knife of media codec libraries, is Free Software regardless of the number of codec patents it breaks. Software does not need any third party approval to be Free Software the rights of Free Software can only be offered by the copyright holders, and the opinion of third parties is an irrelevance as to whether software is Free. The GPL d clone of Blizzard Entertainment s Battle.net servers, bnetd, is Free Software, regardless of legal takedown notices. Third parties cannot influence whether or not a piece of software is Free. They can influence tangentially related topics, such as whether the software can be legally used, but that s the limit. And even within a given piece of software, where copyright is shared by contributors, the author of one component has no say on other components. And you can t make code which is already released as Free, suddenly un-Free you can, if you hold all the copyrights, close up future versions, but your existing code remains Free forever. Reasonably, you can opt to avoid using a piece of software because you have requirements beyond it merely being Free Software Cdrtools has been avoided in Debian for a long time due to the upstream author s legal threats and rambling but that is a side issue as to the question of whether or not the software is Free. Patents are simply not involved in the question of whether or not something is Free Software, except for one narrow case: where Free Software is released by somebody who also holds related patents, and uses a license such as Ms-PL or Apache 2.0 or GPLv3 which requires them to also release those patents to those using/distributing the software. Outside that narrow situation, patents do not relate to the question of whether something is Free Software even if a company releases some source code under a license like BSD then demands patent fees from end users. So, on to the original topic of Mono. Every piece of Mono s source code is released by its authors under a license which guarantees the FSF s Four Freedoms. Whether or not you find Mono useful or tasteful does not affect that Free status. Whether or not Mono infringes upon any laws or patents does not affect that Free status. That Mono contains some libraries whose upstream author is Microsoft does not give Microsoft even the remotest claim to a single line of code outside the code they wrote and even then, it wouldn t be an issue, since the licenses they use are Free. In fact, both the licenses used in the Microsoft portions of the source base make patent grants to all users, in addition to guaranteeing the FSF s Four Freedoms and any license contamination would decrease, not increase, any risk of legal attack from Microsoft. There s even plenty of Microsoft code available for re-use at a lower level than the currently re-used libraries: The Microsoft.NET Micro Framework (for use on embedded platforms direct to the metal) is under the Free Software Apache 2.0 license, and would provide access to some of Microsoft s runtime and class library code, complete with a patent grant, if it were desired. Please try to keep your thought processes straight. If you want to argue that you re all for Free Software, please remember that there s plenty of Free Software you might not approve of and don t claim to be a Free Software advocate then use bogus arguments to claim that Free Software is not Free. Free Software includes LMOS and LOIC and Mono, whether you like it or not.

New feature for Debian CDs We've had a wishlist bug (#551951) open against debian-cd for some time, asking for the creation of "isohybrid" CD images for i386 and amd64. These are special in that as well as the normal CD-based ISO9660 filesystem they also contain a valid-looking DOS-style partition table. Thus, if you simply copy one of these images raw to a USB stick a normal PC BIOS will boot the image directly. This would be a neat feature, making it much easier for people to use standard Debian installer images on their USB sticks without having to follow a lot more instructions. Support for this feature was added in syslinux/isolinux quite a while back (thanks to H Peter Anvin and the other syslinux contributors), but this depended on post-processing images with the isohybrid program. Due to the special way that we make our CD and DVD images to produce jigdo images as well, that wouldn't work for us: the jigdo files would no longer match the ISO files. Bugger...! Recently, along came more help in the form of Thomas Schmitt (author of xorriso) and George Danchev, its Debian maintainer. Recent versions of xorriso support creating isohybrid images directly, which is very handy. We've worked together in the last few months on porting my old JTE code (which creates our jigdo images) from cdrkit to xorriso, and after a lot of testing and debugging we now have things working fully. I've added a small amount of code in debian-cd to use the new xorriso features, and for the last couple of weeks all of the i386, amd64 and i386-amd64 multi-arch CDs and DVDs have been built as hybrids. It's been a delight working with Thomas and George - they're enthusiastic, helpful and friendly. Thanks, guys! What does this mean for end users? Instead of having to specially prepare USB sticks for the installer, you can now simply use dd to write the image straight to the raw stick, e.g.:

# dd if=debian-testing-i386-netinst.iso of=/dev/sdX

The USB stick should now boot directly, showing exactly the same bootup screen as if you'd written the image directly to a CD. Yay! I'm expecting to be using this code from now on for new releases, i.e. d-i beta and RC releases and for Squeeze itself when we get there. I'm not planning on backporting this code into the Lenny branch of debian-cd as it's a lot of work and I don't want to destabilise things there. What's next? We need to clear up some of the maintenance around the code as it stands; I'm planning on moving libjte inside my existing jigit package, as that's a sensible place to put it. That will come after the Squeeze release, though. We need to finalise boot support for other architectures in xorriso; I added support for them into mkisofs/genisoimage a long time ago and we've already moved some of that into xorriso. Thomas and George have also been working hard to give me faster/better size estimation code, useful for making debian-cd run faster. Finally: I'd like to add support into xorriso for creating the nasty HFS hybrid images that are needed for booting Macs. The code that does this in cdrkit is probably some of the worst that I've ever worked with, and I'd like to get away from it. If only Apple hadn't stupidly built their proprietary platform around this shit and had used open standards instead. :-( Once we have these bits in xorriso, I'll be able to move production of all of the CDs across from genisoimage to xorriso. Very nearly there for being able to drop the last remnants of Schily-ware from the Debian archive...

Michael Biebl suggested to me on IRC, that I changed my automated upgrade testing of the Lenny Gnome and KDE Desktop to do apt-get autoremove when using apt-get. This seem like a very good idea, so I adjusted by test scripts and can now present the updated result from today: This is for Gnome: Installed using apt-get, missing with aptitude

apache2.2-bin aptdaemon baobab binfmt-support browser-plugin-gnash cheese-common cli-common cups-pk-helper dmz-cursor-theme empathy empathy-common freedesktop-sound-theme freeglut3 gconf-defaults-service gdm-themes gedit-plugins geoclue geoclue-hostip geoclue-localnet geoclue-manual geoclue-yahoo gnash gnash-common gnome gnome-backgrounds gnome-cards-data gnome-codec-install gnome-core gnome-desktop-environment gnome-disk-utility gnome-screenshot gnome-search-tool gnome-session-canberra gnome-system-log gnome-themes-extras gnome-themes-more gnome-user-share gstreamer0.10-fluendo-mp3 gstreamer0.10-tools gtk2-engines gtk2-engines-pixbuf gtk2-engines-smooth hamster-applet libapache2-mod-dnssd libapr1 libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap libart2.0-cil libboost-date-time1.42.0 libboost-python1.42.0 libboost-thread1.42.0 libchamplain-0.4-0 libchamplain-gtk-0.4-0 libcheese-gtk18 libclutter-gtk-0.10-0 libcryptui0 libdiscid0 libelf1 libepc-1.0-2 libepc-common libepc-ui-1.0-2 libfreerdp-plugins-standard libfreerdp0 libgconf2.0-cil libgdata-common libgdata7 libgdu-gtk0 libgee2 libgeoclue0 libgexiv2-0 libgif4 libglade2.0-cil libglib2.0-cil libgmime2.4-cil libgnome-vfs2.0-cil libgnome2.24-cil libgnomepanel2.24-cil libgpod-common libgpod4 libgtk2.0-cil libgtkglext1 libgtksourceview2.0-common libmono-addins-gui0.2-cil libmono-addins0.2-cil libmono-cairo2.0-cil libmono-corlib2.0-cil libmono-i18n-west2.0-cil libmono-posix2.0-cil libmono-security2.0-cil libmono-sharpzip2.84-cil libmono-system2.0-cil libmtp8 libmusicbrainz3-6 libndesk-dbus-glib1.0-cil libndesk-dbus1.0-cil libopal3.6.8 libpolkit-gtk-1-0 libpt2.6.7 libpython2.6 librpm1 librpmio1 libsdl1.2debian libsrtp0 libssh-4 libtelepathy-farsight0 libtelepathy-glib0 libtidy-0.99-0 media-player-info mesa-utils mono-2.0-gac mono-gac mono-runtime nautilus-sendto nautilus-sendto-empathy p7zip-full pkg-config python-aptdaemon python-aptdaemon-gtk python-axiom python-beautifulsoup python-bugbuddy python-clientform python-coherence python-configobj python-crypto python-cupshelpers python-elementtree python-epsilon python-evolution python-feedparser python-gdata python-gdbm python-gst0.10 python-gtkglext1 python-gtksourceview2 python-httplib2 python-louie python-mako python-markupsafe python-mechanize python-nevow python-notify python-opengl python-openssl python-pam python-pkg-resources python-pyasn1 python-pysqlite2 python-rdflib python-serial python-tagpy python-twisted-bin python-twisted-conch python-twisted-core python-twisted-web python-utidylib python-webkit python-xdg python-zope.interface remmina remmina-plugin-data remmina-plugin-rdp remmina-plugin-vnc rhythmbox-plugin-cdrecorder rhythmbox-plugins rpm-common rpm2cpio seahorse-plugins shotwell software-center system-config-printer-udev telepathy-gabble telepathy-mission-control-5 telepathy-salut tomboy totem totem-coherence totem-mozilla totem-plugins transmission-common xdg-user-dirs xdg-user-dirs-gtk xserver-xephyr

Installed using apt-get, removed with aptitude

cheese ekiga eog epiphany-extensions evolution-exchange fast-user-switch-applet file-roller gcalctool gconf-editor gdm gedit gedit-common gnome-games gnome-games-data gnome-nettool gnome-system-tools gnome-themes gnuchess gucharmap guile-1.8-libs libavahi-ui0 libdmx1 libgalago3 libgtk-vnc-1.0-0 libgtksourceview2.0-0 liblircclient0 libsdl1.2debian-alsa libspeexdsp1 libsvga1 rhythmbox seahorse sound-juicer system-config-printer totem-common transmission-gtk vinagre vino

Installed using aptitude, missing with apt-get

gstreamer0.10-gnomevfs

Installed using aptitude, removed with apt-get

[nothing]

This is for KDE: Installed using apt-get, missing with aptitude

ksmserver

Installed using apt-get, removed with aptitude

kwin network-manager-kde

Installed using aptitude, missing with apt-get

arts dolphin freespacenotifier google-gadgets-gst google-gadgets-xul kappfinder kcalc kcharselect kde-core kde-plasma-desktop kde-standard kde-window-manager kdeartwork kdeartwork-emoticons kdeartwork-style kdeartwork-theme-icon kdebase kdebase-apps kdebase-workspace kdebase-workspace-bin kdebase-workspace-data kdeeject kdelibs kdeplasma-addons kdeutils kdewallpapers kdf kfloppy kgpg khelpcenter4 kinfocenter konq-plugins-l10n konqueror-nsplugins kscreensaver kscreensaver-xsavers ktimer kwrite libgle3 libkde4-ruby1.8 libkonq5 libkonq5-templates libnetpbm10 libplasma-ruby libplasma-ruby1.8 libqt4-ruby1.8 marble-data marble-plugins netpbm nuvola-icon-theme plasma-dataengines-workspace plasma-desktop plasma-desktopthemes-artwork plasma-runners-addons plasma-scriptengine-googlegadgets plasma-scriptengine-python plasma-scriptengine-qedje plasma-scriptengine-ruby plasma-scriptengine-webkit plasma-scriptengines plasma-wallpapers-addons plasma-widget-folderview plasma-widget-networkmanagement ruby sweeper update-notifier-kde xscreensaver-data-extra xscreensaver-gl xscreensaver-gl-extra xscreensaver-screensaver-bsod

Installed using aptitude, removed with apt-get

ark google-gadgets-common google-gadgets-qt htdig kate kdebase-bin kdebase-data kdepasswd kfind klipper konq-plugins konqueror ksysguard ksysguardd libarchive1 libcln6 libeet1 libeina-svn-06 libggadget-1.0-0b libggadget-qt-1.0-0b libgps19 libkdecorations4 libkephal4 libkonq4 libkonqsidebarplugin4a libkscreensaver5 libksgrd4 libksignalplotter4 libkunitconversion4 libkwineffects1a libmarblewidget4 libntrack-qt4-1 libntrack0 libplasma-geolocation-interface4 libplasmaclock4a libplasmagenericshell4 libprocesscore4a libprocessui4a libqalculate5 libqedje0a libqtruby4shared2 libqzion0a libruby1.8 libscim8c2a libsmokekdecore4-3 libsmokekdeui4-3 libsmokekfile3 libsmokekhtml3 libsmokekio3 libsmokeknewstuff2-3 libsmokeknewstuff3-3 libsmokekparts3 libsmokektexteditor3 libsmokekutils3 libsmokenepomuk3 libsmokephonon3 libsmokeplasma3 libsmokeqtcore4-3 libsmokeqtdbus4-3 libsmokeqtgui4-3 libsmokeqtnetwork4-3 libsmokeqtopengl4-3 libsmokeqtscript4-3 libsmokeqtsql4-3 libsmokeqtsvg4-3 libsmokeqttest4-3 libsmokeqtuitools4-3 libsmokeqtwebkit4-3 libsmokeqtxml4-3 libsmokesolid3 libsmokesoprano3 libtaskmanager4a libtidy-0.99-0 libweather-ion4a libxklavier16 libxxf86misc1 okteta oxygencursors plasma-dataengines-addons plasma-scriptengine-superkaramba plasma-widget-lancelot plasma-widgets-addons plasma-widgets-workspace polkit-kde-1 ruby1.8 systemsettings update-notifier-common

Running apt-get autoremove made the results using apt-get and aptitude a bit more similar, but there are still quite a lott of differences. I have no idea what packages should be installed after the upgrade, but hope those that do can have a look.

I'm still running upgrade testing of the Lenny Gnome and KDE Desktop, but have not had time to spend on reporting the status. Here is a short update based on a test I ran 20101118. I still do not know what a correct migration should look like, so I report any differences between apt and aptitude and hope someone else can see if anything should be changed. This is for Gnome: Installed using apt-get, missing with aptitude

apache2.2-bin aptdaemon at-spi baobab binfmt-support browser-plugin-gnash cheese-common cli-common cpp-4.3 cups-pk-helper dmz-cursor-theme empathy empathy-common finger freedesktop-sound-theme freeglut3 gconf-defaults-service gdm-themes gedit-plugins geoclue geoclue-hostip geoclue-localnet geoclue-manual geoclue-yahoo gnash gnash-common gnome gnome-backgrounds gnome-cards-data gnome-codec-install gnome-core gnome-desktop-environment gnome-disk-utility gnome-screenshot gnome-search-tool gnome-session-canberra gnome-spell gnome-system-log gnome-themes-extras gnome-themes-more gnome-user-share gs-common gstreamer0.10-fluendo-mp3 gstreamer0.10-tools gtk2-engines gtk2-engines-pixbuf gtk2-engines-smooth hal-info hamster-applet libapache2-mod-dnssd libapr1 libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap libart2.0-cil libatspi1.0-0 libboost-date-time1.42.0 libboost-python1.42.0 libboost-thread1.42.0 libchamplain-0.4-0 libchamplain-gtk-0.4-0 libcheese-gtk18 libclutter-gtk-0.10-0 libcryptui0 libcupsys2 libdiscid0 libeel2-data libelf1 libepc-1.0-2 libepc-common libepc-ui-1.0-2 libfreerdp-plugins-standard libfreerdp0 libgail-common libgconf2.0-cil libgdata-common libgdata7 libgdl-1-common libgdu-gtk0 libgee2 libgeoclue0 libgexiv2-0 libgif4 libglade2.0-cil libglib2.0-cil libgmime2.4-cil libgnome-vfs2.0-cil libgnome2.24-cil libgnomepanel2.24-cil libgnomeprint2.2-data libgnomeprintui2.2-common libgnomevfs2-bin libgpod-common libgpod4 libgtk2.0-cil libgtkglext1 libgtksourceview-common libgtksourceview2.0-common libmono-addins-gui0.2-cil libmono-addins0.2-cil libmono-cairo2.0-cil libmono-corlib2.0-cil libmono-i18n-west2.0-cil libmono-posix2.0-cil libmono-security2.0-cil libmono-sharpzip2.84-cil libmono-system2.0-cil libmtp8 libmusicbrainz3-6 libndesk-dbus-glib1.0-cil libndesk-dbus1.0-cil libopal3.6.8 libpolkit-gtk-1-0 libpt-1.10.10-plugins-alsa libpt-1.10.10-plugins-v4l libpt2.6.7 libpython2.6 librpm1 librpmio1 libsdl1.2debian libservlet2.4-java libsrtp0 libssh-4 libtelepathy-farsight0 libtelepathy-glib0 libtidy-0.99-0 libxalan2-java libxerces2-java media-player-info mesa-utils mono-2.0-gac mono-gac mono-runtime nautilus-sendto nautilus-sendto-empathy openoffice.org-writer2latex openssl-blacklist p7zip p7zip-full pkg-config python-4suite-xml python-aptdaemon python-aptdaemon-gtk python-axiom python-beautifulsoup python-bugbuddy python-clientform python-coherence python-configobj python-crypto python-cupshelpers python-cupsutils python-eggtrayicon python-elementtree python-epsilon python-evolution python-feedparser python-gdata python-gdbm python-gst0.10 python-gtkglext1 python-gtkmozembed python-gtksourceview2 python-httplib2 python-louie python-mako python-markupsafe python-mechanize python-nevow python-notify python-opengl python-openssl python-pam python-pkg-resources python-pyasn1 python-pysqlite2 python-rdflib python-serial python-tagpy python-twisted-bin python-twisted-conch python-twisted-core python-twisted-web python-utidylib python-webkit python-xdg python-zope.interface remmina remmina-plugin-data remmina-plugin-rdp remmina-plugin-vnc rhythmbox-plugin-cdrecorder rhythmbox-plugins rpm-common rpm2cpio seahorse-plugins shotwell software-center svgalibg1 system-config-printer-udev telepathy-gabble telepathy-mission-control-5 telepathy-salut tomboy totem totem-coherence totem-mozilla totem-plugins transmission-common xdg-user-dirs xdg-user-dirs-gtk xserver-xephyr zip

Installed using apt-get, removed with aptitude

arj bluez-utils cheese dhcdbd djvulibre-desktop ekiga eog epiphany-extensions epiphany-gecko evolution-exchange fast-user-switch-applet file-roller gcalctool gconf-editor gdm gedit gedit-common gnome-app-install gnome-games gnome-games-data gnome-nettool gnome-system-tools gnome-themes gnome-utils gnome-vfs-obexftp gnome-volume-manager gnuchess gucharmap guile-1.8-libs hal libavahi-compat-libdnssd1 libavahi-core5 libavahi-ui0 libbind9-50 libbluetooth2 libcamel1.2-11 libcdio7 libcucul0 libcurl3 libdirectfb-1.0-0 libdmx1 libdvdread3 libedata-cal1.2-6 libedataserver1.2-9 libeel2-2.20 libepc-1.0-1 libepc-ui-1.0-1 libexchange-storage1.2-3 libfaad0 libgadu3 libgalago3 libgd2-noxpm libgda3-3 libgda3-common libggz2 libggzcore9 libggzmod4 libgksu1.2-0 libgksuui1.0-1 libgmyth0 libgnome-desktop-2 libgnome-pilot2 libgnomecups1.0-1 libgnomeprint2.2-0 libgnomeprintui2.2-0 libgpod3 libgraphviz4 libgtk-vnc-1.0-0 libgtkhtml2-0 libgtksourceview1.0-0 libgtksourceview2.0-0 libgucharmap6 libhesiod0 libicu38 libisccc50 libisccfg50 libiw29 libjaxp1.3-java-gcj libkpathsea4 liblircclient0 libltdl3 liblwres50 libmagick++10 libmagick10 libmalaga7 libmozjs1d libmpfr1ldbl libmtp7 libmysqlclient15off libnautilus-burn4 libneon27 libnm-glib0 libnm-util0 libopal-2.2 libosp5 libparted1.8-10 libpisock9 libpisync1 libpoppler-glib3 libpoppler3 libpt-1.10.10 libraw1394-8 libsdl1.2debian-alsa libsensors3 libsexy2 libsmbios2 libsoup2.2-8 libspeexdsp1 libssh2-1 libsuitesparse-3.1.0 libsvga1 libswfdec-0.6-90 libtalloc1 libtotem-plparser10 libtrackerclient0 libvoikko1 libxalan2-java-gcj libxerces2-java-gcj libxklavier12 libxtrap6 libxxf86misc1 libzephyr3 mysql-common rhythmbox seahorse sound-juicer swfdec-gnome system-config-printer totem-common totem-gstreamer transmission-gtk vinagre vino w3c-dtd-xhtml wodim

Installed using aptitude, missing with apt-get

gstreamer0.10-gnomevfs

Installed using aptitude, removed with apt-get

[nothing]

This is for KDE: Installed using apt-get, missing with aptitude

autopoint bomber bovo cantor cantor-backend-kalgebra cpp-4.3 dcoprss edict espeak espeak-data eyesapplet fifteenapplet finger gettext ghostscript-x git gnome-audio gnugo granatier gs-common gstreamer0.10-pulseaudio indi kaddressbook-plugins kalgebra kalzium-data kanjidic kapman kate-plugins kblocks kbreakout kbstate kde-icons-mono kdeaccessibility kdeaddons-kfile-plugins kdeadmin-kfile-plugins kdeartwork-misc kdeartwork-theme-window kdeedu kdeedu-data kdeedu-kvtml-data kdegames kdegames-card-data kdegames-mahjongg-data kdegraphics-kfile-plugins kdelirc kdemultimedia-kfile-plugins kdenetwork-kfile-plugins kdepim-kfile-plugins kdepim-kio-plugins kdessh kdetoys kdewebdev kdiamond kdnssd kfilereplace kfourinline kgeography-data kigo killbots kiriki klettres-data kmoon kmrml knewsticker-scripts kollision kpf krosspython ksirk ksmserver ksquares kstars-data ksudoku kubrick kweather libasound2-plugins libboost-python1.42.0 libcfitsio3 libconvert-binhex-perl libcrypt-ssleay-perl libdb4.6++ libdjvulibre-text libdotconf1.0 liberror-perl libespeak1 libfinance-quote-perl libgail-common libgsl0ldbl libhtml-parser-perl libhtml-tableextract-perl libhtml-tagset-perl libhtml-tree-perl libio-stringy-perl libkdeedu4 libkdegames5 libkiten4 libkpathsea5 libkrossui4 libmailtools-perl libmime-tools-perl libnews-nntpclient-perl libopenbabel3 libportaudio2 libpulse-browse0 libservlet2.4-java libspeechd2 libtiff-tools libtimedate-perl libunistring0 liburi-perl libwww-perl libxalan2-java libxerces2-java lirc luatex marble networkstatus noatun-plugins openoffice.org-writer2latex palapeli palapeli-data parley parley-data poster psutils pulseaudio pulseaudio-esound-compat pulseaudio-module-x11 pulseaudio-utils quanta-data rocs rsync speech-dispatcher step svgalibg1 texlive-binaries texlive-luatex ttf-sazanami-gothic

Installed using apt-get, removed with aptitude

amor artsbuilder atlantik atlantikdesigner blinken bluez-utils cvs dhcdbd djvulibre-desktop imlib-base imlib11 kalzium kanagram kandy kasteroids katomic kbackgammon kbattleship kblackbox kbounce kbruch kcron kdat kdemultimedia-kappfinder-data kdeprint kdict kdvi kedit keduca kenolaba kfax kfaxview kfouleggs kgeography kghostview kgoldrunner khangman khexedit kiconedit kig kimagemapeditor kitchensync kiten kjumpingcube klatin klettres klickety klines klinkstatus kmag kmahjongg kmailcvt kmenuedit kmid kmilo kmines kmousetool kmouth kmplot knetwalk kodo kolf kommander konquest kooka kpager kpat kpdf kpercentage kpilot kpoker kpovmodeler krec kregexpeditor kreversi ksame ksayit kshisen ksig ksim ksirc ksirtet ksmiletris ksnake ksokoban kspaceduel kstars ksvg ksysv kteatime ktip ktnef ktouch ktron kttsd ktuberling kturtle ktux kuickshow kverbos kview kviewshell kvoctrain kwifimanager kwin kwin4 kwordquiz kworldclock kxsldbg libakode2 libarts1-akode libarts1-audiofile libarts1-mpeglib libarts1-xine libavahi-compat-libdnssd1 libavahi-core5 libavc1394-0 libbind9-50 libbluetooth2 libboost-python1.34.1 libcucul0 libcurl3 libcvsservice0 libdirectfb-1.0-0 libdjvulibre21 libdvdread3 libfaad0 libfreebob0 libgd2-noxpm libgraphviz4 libgsmme1c2a libgtkhtml2-0 libicu38 libiec61883-0 libindex0 libisccc50 libisccfg50 libiw29 libjaxp1.3-java-gcj libk3b3 libkcal2b libkcddb1 libkdeedu3 libkdegames1 libkdepim1a libkgantt0 libkleopatra1 libkmime2 libkpathsea4 libkpimexchange1 libkpimidentities1 libkscan1 libksieve0 libktnef1 liblockdev1 libltdl3 liblwres50 libmagick10 libmimelib1c2a libmodplug0c2 libmozjs1d libmpcdec3 libmpfr1ldbl libneon27 libnm-util0 libopensync0 libpisock9 libpoppler-glib3 libpoppler-qt2 libpoppler3 libraw1394-8 librss1 libsensors3 libsmbios2 libssh2-1 libsuitesparse-3.1.0 libswfdec-0.6-90 libtalloc1 libxalan2-java-gcj libxerces2-java-gcj libxtrap6 lskat mpeglib network-manager-kde noatun pmount tex-common texlive-base texlive-common texlive-doc-base texlive-fonts-recommended tidy ttf-dustin ttf-kochi-gothic ttf-sjfonts

Installed using aptitude, missing with apt-get

dolphin kde-core kde-plasma-desktop kde-standard kde-window-manager kdeartwork kdebase kdebase-apps kdebase-workspace kdebase-workspace-bin kdebase-workspace-data kdeutils kscreensaver kscreensaver-xsavers libgle3 libkonq5 libkonq5-templates libnetpbm10 netpbm plasma-widget-folderview plasma-widget-networkmanagement xscreensaver-data-extra xscreensaver-gl xscreensaver-gl-extra xscreensaver-screensaver-bsod

Installed using aptitude, removed with apt-get

kdebase-bin konq-plugins konqueror

/etc/fstab files tend to be an unreadable mess of unaligned fields.

# /etc/fstab: static file system information.
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    defaults        0       0
/dev/mapper/benz-root /               ext3    errors=remount-ro 0       1
/dev/sda1       /boot           ext3    defaults        0       2
/dev/mapper/benz-home /home           ext3    defaults        0       2
/dev/mapper/benz-swap_1 none            swap    sw              0       0
newton:/home        /nfs        nfs defaults,soft,intr,users    0 0
/dev/scd0       /media/cdrom0   udf,iso9660 user,noauto     0       0

Let's remove some whitespace in the third line:

#<filesystem> <mountpoint>   <type>  <options>       <dump>  <pass>

And then pipe everything from line 3 to the end through column -t:

# /etc/fstab: static file system information.
#
#<filesystem>            <mountpoint>   <type>       <options>                 <dump>  <pass>
proc                     /proc          proc         defaults                  0       0
/dev/mapper/benz-root    /              ext3         errors=remount-ro         0       1
/dev/sda1                /boot          ext3         defaults                  0       2
/dev/mapper/benz-home    /home          ext3         defaults                  0       2
/dev/mapper/benz-swap_1  none           swap         sw                        0       0
newton:/home             /nfs           nfs          defaults,soft,intr,users  0       0
/dev/scd0                /media/cdrom0  udf,iso9660  user,noauto               0       0

Thanks to SP8472 for bringing this to my attention.

You have a computer. You're afraid of it being stolen by baddies or raided by the police, so you've encrypted its hard disk with LUKS. You also want to carry around some of your crypto keys (SSH and/or GPG), or any kind of sensitive data, on an USB key, so you can restore a normal activity in case the computer gets stolen or becomes untrustworthy. You like things that Just Work, including automounting with autofs (no GUI popups, no need to manually unmount and so on). But of course, you can't very well just mount the drive and store your keys on it in the clear. Well, actually you can (I did it for way too long), but let's assume you want to do without. Because USB keys get stolen or lost, too. autofs just handles the mounting of the drives, not their unlocking, and there's no hook in there specifically for LUKS. There is, however, a more generic way of running arbitrary commands on mounting: the program maps. It's all described in the documentation, so I'll just paste the relevant parts of my /etc/auto.removable, with a few comments. Remember to make it executable and reference it in /etc/auto.master.

 #! /bin/sh
 volume=$1
 autoluks ()  
   dev=/dev/$1
   cryptname=$ 1 _crypt
   cryptdev=/dev/mapper/$cryptname
   keyfile=/etc/private/$cryptname.key
   options=$2
   [ ! -e $dev ] && exit 0
   if [ -b $cryptdev ] && ! cryptsetup status $cryptname   grep -q device:.*$dev ; then
       cryptsetup remove $cryptname   logger
   fi
   cryptsetup --key-file $keyfile luksOpen $dev $cryptname   logger
   [ -b $cryptdev ] && echo $options :$cryptdev   true
  
 case "$volume" in
   # LUKS-encrypted volumes
   lacie-keys)
       autoluks lacie-keys -fstype=ext4,ro,noatime,nodev,noexec,nosuid
       ;;
   lacie-keys-rw)
       autoluks lacie-keys -fstype=ext4
       ;;
   lacie-backups)
       autoluks lacie-backups -fstype=ext4
       ;;
   # Non-encrypted volumes
   cd)
       echo -fstype=iso9660,ro,nodev,nosuid            :/dev/cdrom
       ;;
   floppy)
       echo -fstype=auto,sync,nodev,nosuid             :/dev/fd0
       ;;
 esac

The astute reader will have noticed that the keys volume is mounted either read-only or read-write with the same mechanism. This volume contains SSH keys that I use quite regularly, that filesystem is going to be mounted and unmounted very often. USB keys, and flash memory in general, don't really like repeated writes. Fortunately, the SSH keys are mostly read, and very rarely written to, so I can afford to mount the partition read-only and save the write cycles for when I generate new keys (the mere action of mounting a partition read-write changes it, even if the files in it are never modified). The backups volume is mostly accessed in write mode, and much less frequently anyway, so there's no need for a distinction there. This supposes that the partitions on the USB key appear as /dev/lacie-keys and /dev/lacie-backups. Due to the dynamic naming of devices, you may like to use rules similar to the following in /etc/udev/rules.d/local.

 KERNEL=="sd[a-z]1", ATTRS idProduct =="1027", ATTRS idVendor =="059f", ATTRS manufacturer =="LaCie", ATTRS product =="LaCie iamaKey", SYMLINK+="lacie-keys"
 KERNEL=="sd[a-z]2", ATTRS idProduct =="1027", ATTRS idVendor =="059f", ATTRS manufacturer =="LaCie", ATTRS product =="LaCie iamaKey", SYMLINK+="lacie-backups"

You'll also need to initialize LUKS on the partitions, save the key, make the filesystems, and so on. If you're still with me, you probably know what I'm talking about, then I don't need to explain about that. This setup works for me, but there's no guarantee and so on. Based on a tutorial found on the Debian Administration website (great resource, by the way). Update: I'm told a large part of the script could be replaced with pmount. If you like short scripts, you may want to investigate that too. Update 2: Josselin points out a different solution using Gnome, udisks, gnome-disk-utility and Nautilus. It's worth checking out, for those who like to run all that and click around. I can't test it right now (Gnome/Nautilus currently refuses to mount any USB key, for some presumably transitional reason), but I have no reason to believe it's inherently broken. I just don't like having to do anything when I plug my key or before I unplug it, and I'm not always running a graphical session with a desktop.

Given the impending deprecation of vservers, I've decided to make the switch to KVM on my laptop. Although lxc is a closer approximation to vservers, I decided to go with KVM due to it's support in Virtual Machine Manager. My first step was to confirm that my CPU would support kvm:

egrep -o "svm vmx" /proc/cpuinfo

If that command outputs either svm or vmx (depending on whether you have Intel or AMD hardware) then your CPU supports virtualization. I'm working on a host machine called chicken, which has a logical volume called vg_chicken0. All vservers on chicken operate on a root filesystem that is backed by their own logical volume. In this post, I'll describe the steps to convert the vserver hobo (which operates on a filesystem mounted on the host in /var/lib/vservers/hobo and is backed by the logical volume called vg_chicken0-hobo_root). Both chicken and hobo are running debian squeeze. vservers don't have a kernel installed or grub. KVM virtual servers need both. I was hoping I could simply enter the vserver, install both a kernal and grub and be ready to go. However, grub installation will fail miserably because grub can't figure out how to install on the underlying disk (which is hidden from the vserver). Next, I tried launching a kvm instance, passing a debirf generated ISO with the -c (cdrom) option. However, grub recognized that it was being installed onto a device that did not have a partition table (the logical volume was directly formatted with a file system). So, since I had disk space to spare, I created a new logical volume:

lvcreate --size 5GB --name hobo_root_new vg_chicken0

I then added a gpt partition table (why not prepare for the coming 2TB disks?) and created two partitions. One partition for grub2 and one for everything else:

parted /dev/mapper/vg_chicken0-hobo_root_new mklabel gpt
parted /dev/mapper/vg_chicken0-hobo_root_new unit s mkpart biosboot 2048 4095 
parted /dev/mapper/vg_chicken0-hobo_root_new set 1 bios_grub on 
parted /dev/mapper/vg_chicken0-hobo_root_new unit s mkpart primary 4096

When prompted for the end of the last partition, choose: -1 and accept the adjustment. I had to eyeball cat /proc/partitions to figure out which dm device was the second partition (dm-19). I then created a file system:

mkfs -t ext3 /dev/dm-19

Mounted it:

mount /dev/dm-19 /mnt

And rsync'ed the data:

rsync -a /var/lib/vservers/hobo/ /mnt/

With the data in place, I chroot'ed and installed the packages I needed. When prompted, I chose not to install grub to the disk, because I wanted to wait until I had an environment in which the proper disk would be available to grub as it will when the virtual server boots (see below):

chroot /mnt
mount /proc
aptitude install linux-image-2.6-amd64 grub2
umount /proc
exit

Then, I cleaned up:

umount /mnt
umount /var/lib/vservers/hobo
lvremove vg_chicken0/hobo_root
lvrename vg_chicken0/hobo_root_new hobo_root
dmsetup remove /dev/mapper/vg_chicken0-hobo_root_newp1
dmsetup remove /dev/mapper/vg_chicken0-hobo_root_newp2
kpartx -d /dev/mapper/vg_chicken0-hobo_root

And I removed it from /etc/fstab. Next, I created a new kvm virtual server, using the disk /dev/mapper/vg_chicken0-hobo_root and passing a debirf cd image with -c: virt-install --name hobo --ram 512 --disk /dev/mapper/vgchicken0-hoboroot -c /usr/local/share/debian/ISOs/debirf-rescuesqueeze2.6.32-5-vserver-amd64.iso After logging in, I installed grub2 (aptitude update; aptitude install grub2) and then I installed grub:

mount /dev/sda2 /mnt/
grub-install --no-floppy --root-directory=/mnt /dev/sda

After running grub-install, edit /mnt/boot/grub/device.map so it reads:

(hd0) /dev/sda

Then, rerun grub-install command. I tried generating the grub.cfg file, but got an error message indicating that grub-probe would not detect the device providing / (because I was running on a ram file system from debirf). I added the following to /mnt/etc/fstab:

/dev/sda2  /     ext3 errors=remount-ro 0 1
proc       /proc proc defaults          0 0

And then re-generate the initrd image:

chroot /mnt
mount /proc
update-initramfs -u

So, I rebooted the virtual machine by typing:

exit
reboot

This dropped me into a grub shell. I manually typed:

root (hd0,gpt2)
linux /vmlinuz root=/dev/sda2 ro
initrd /initrd.img
boot

Once booted, I logged in a completed the task with:

update-grub

This blog post is a kind of followup of a previous blog post about reading 10 years old CD-R. Today, I get a new shiny ASRock ION 330 to play with, in itself this PC has nothing particular. As usual, I begin by using System Rescue CD to check bad blocks on the hard drive, before working with it (badblocks -v -s -w -c 4096 /dev/sda). Once done, I will check the memory (memtest). This is my standard procedure when receiving a new PC. But I don't have my System Rescue CD at hand. No problem, I download it and and write it on an old CD-RW, bought 5 years ago but still sealed into his blister. For your information, this is a Memorex CD-RW 700MB 16-24x. I blank it (wodim blank=all). I write it at speed 16x, because I don't know how to lower the speed. After writing the CD, I re-read it to check everything is fine (readom dev=/dev/cdrw1 f=test.iso). Ooops:

$ readom dev=/dev/cdrw1 f=test.iso
[...] 
Errno: 5 (Input/output error), read_g1 scsi sendcmd: no error
CDB:  28 00 00 01 3C 00 00 00 40 00
status: 0x2 (CHECK CONDITION)
Sense Bytes: 70 00 03 00 00 00 00 0A 00 00 00 00 11 05 00 00
Sense Key: 0x3 Medium Error, Segment 0
Sense Code: 0x11 Qual 0x05 (l-ec uncorrectable error) Fru 0x0
Sense flags: Blk 0 (not valid) 
cmd finished after 6.614s timeout 40s
readom: Input/output error. Cannot read source disk
readom: Retrying from sector 80896.
.............
readom: Error on sector 80908 corrected after 9 tries. Total of 0 errors.
..............---

OK, I re-blank it (wodim blank=fast) and rewrite it. The CD seems to be readable this time. But when I try to boot with it my new ASRock, it fails. Enough is enough, I get an heavy used Verbatim 8cm CD-RW. Just to show you how old he is: the first ISO written on it was "Debian Sarge". And the magic happens: it works on the first try!!!! You will probably think that I work for Verbatim. This is not true but I must admit that first-price CD-RW/CD-R seems to be very low quality and don't hold data in time. This is probably related to the chemical compounds of the CD. The failing Memorex CD-RW will end up in a trash and I will continue to use my 5 years old Verbatim CD-RW. Dear reader, in order to find a way not to dump this failing CD-RW, do you know a way to make it to work? What you should now about the failing CD-RW:

• there are no scratches on the surface
• the CD-RW has never been used before this morning
• I keep it out of the direct sunlight
• it was stored next to the Verbatim CD-RW

After a long break from debian-installer development, I finally found time today to return to the project. Having to spend less time working dependency based boot in debian, as it is almost complete now, definitely helped freeing some time. A while back, I ran into a problem while working on Debian Edu. We include some firmware packages on the Debian Edu CDs, those needed to get disk and network controllers working. Without having these firmware packages available during installation, it is impossible to install Debian Edu on the given machine, and because our target group are non-technical people, asking them to provide firmware packages on an external medium is a support pain. Initially, I expected it to be enough to include the firmware packages on the CD to get debian-installer to find and use them. This proved to be wrong. Next, I hoped it was enough to symlink the relevant firmware packages to some useful location on the CD (tried /cdrom/ and /cdrom/firmware/). This also proved to not work, and at this point I found time to look at the debian-installer code to figure out what was going to work. The firmware loading code is in the hw-detect package, and a closer look revealed that it would only look for firmware packages outside the installation media, so the CD was never checked for firmware packages. It would only check USB sticks, floppies and other "external" media devices. Today I changed it to also look in the /cdrom/firmware/ directory on the mounted CD or DVD, which should solve the problem I ran into with Debian edu. I also changed it to look in /firmware/, to make sure the installer also find firmware provided in the initrd when booting the installer via PXE, to allow us to provide the same feature in the PXE setup included in Debian Edu. To make sure firmware deb packages with a license questions are not activated without asking if the license is accepted, I extended hw-detect to look for preinst scripts in the firmware packages, and run these before activating the firmware during installation. The license question is asked using debconf in the preinst, so this should solve the issue for the firmware packages I have looked at so far. If you want to discuss the details of these features, please contact us on debian-boot@lists.debian.org.

C. Guy Yarvin is a good friend of Mencius Moldbug, a pseudonymous blogger known for iconoclastic novella-length essays on politics and history (and occasionally computer science). Guy recently published under his own name a novel project in language and systems design. His own writing about his work is entertaining but verbose (as Moldbug's readers might expect), so I will attempt to summarize it here. Nock, Urbit, Watt First there is Nock, a tool for defining higher-level languages comparable to the lambda calculus, but meant as foundational system software rather than foundational meta mathe matics. Its primitives include positive integers with equality and increment operators, cons cells with car/cdr/cadr/etc., and a macro for convenient branching. Nock uses trees of integers to represent both code and data. Next, Guy provides the rationale for Nock. In short, he asks how a planet-wide computing infrastructure (OS, networking, and languages) would look if designed from first priniciples for robustness and interoperability. The answer he proposes is Urbit: a URI-like name space distributed globally via content-centric networking, with a feudal structure for top-level names and cryptographic identities. Urbit is a static functional name space: it is both referentially transparent and monotonic (a name, once bound to a value, cannot be un- or re-bound). Why does this require a new formal logic and a new programming language? In Urbit, all data and code are distributed via the global namespace. For interoperability, the code must have a standard format. Nock's minimal spec is meant to be an un ambiguous, unchanging, totally standardized basis for computation in Urbit. Above it will be Watt, a self-hosting language that compiles to Nock. Urbit itself will be implemented in Watt, so Nock and Watt are designed to treat data as code using metacircular evaluation. The code A prototype implementation of Watt is on GitHub. It is not yet self-hosting; the current compiler is written in C. Watt is a functional language with static types called molds and a mechanism for explicit lazy evaluation. (I was suprised to find I had accidentally created an in com patible lazy dialect of Nock despite its goal of unambiguous semantics just by implementing it in Haskell.) The code is not fully documented, but the repository contains draft specs for both Watt and Urbit. Beware: the syntax and terminology are a bit unconventional. Guy has offered a few exercises to help get started with Nock and Watt:

The Nock challenge:

Write a decrement operator in Nock, and an interpreter that can evaluate it.

Basic Watt:

Write an integer square root function in Watt.

Advanced Watt:

How would you write a function that tests whether molds A and B are orthogonal (no noun is in both A and B)? Or compatible (any noun in A is also in B)? Are these functions NP-complete? If so, how might one work around this in practice?

If you want to learn more, start with these problems. You can email your solutions to Guy. Will it work? I find Urbit intellectually appealing; it is a simple and clean architecture that could potentially replace a lot of complex system software. But can we get there from here? Guy imagines Urbit as the product of an ages-old Martian civilization:

Since Earth code is fifty years old, and Martian code is fifty million years old, Martian code has been evolving into a big ball of mud for a million times longer than Earth software. (And two million times longer than Windows.) Therefore, at some point in Martian history, some abject fsck of a Martian code-monkey must have said: fsck this entire fscking ball of mud. For lo, its defects cannot be summarized; for they exceed the global supply of bullet points; for numerous as the fishes in the sea, like the fishes in the sea they fsck, making more little fscking fishes. For lo, it is fscked, and a big ball of mud. And there is only one thing to do with it: obliterate the trunk, fire the developers, and hire a whole new fscking army of Martian code-monkeys to rewrite the entire fscking thing. This is the crucial inference we can draw about Mars: since the Martians had 50 million years to try, in the end they must have succeeded. The result: Martian code, as we know it today. Not enormous and horrible tiny and diamond-perfect. Moreover, because it is tiny and diamond-perfect, it is perfectly stable and never changes or decays. It neither is a big ball of mud, nor tends to become one. It has achieved its final, permanent and excellent state.

Do Earthlings have the will to throw out the whole ball of mud and start from scratch? I doubt it. We can build Urbit but no one will come, unless it solves some problem radically better than current software. Moldbug thinks feudalism will produce better online reputation, but feudal reputation does not require feudal identity; it is not that much harder to build Moldbug's reputation system on Earth than on Mars. I still have not figured out the killer app that will get early adopters to switch to Urbit.

As some people noticed, a driver update to support a major hardware release in my day job and a long series of trivial patches motivated by Debian kernel work combined to make me joint most prolific change author for Linux 2.6.33. This was a fluke, and 2.6.34 is likely to be fairly quiet for me. Team work Meanwhile the kernel team is still working hard on fixing bugs in Linux 2.6.32. As previously announced, this kernel version will be used in both Debian 6.0 'squeeze' and Ubuntu 10.04 LTS. It is also known to be the kernel version for RHEL 6 though this has not been officially announced. If you find a serious bug that was fixed in 2.6.33 or later, please ask the author to send the fix to stable@kernel.org so that it will be included in all these distributions. Note that this happens automatically for commits with the line 'Cc: stable@kernel.org' in their description. We are also continuing to backport new drivers and new hardware support in existing drivers into Debian 5.0 'lenny' (stable). Missing support for common hardware is considered an important bug and suitable for stable updates. However, if there were major changes to a driver before those that added new hardware support, we may be unable to produce a backport. We are not able to carry out comprehensive hardware testing and do not want to risk a regression existing hardware support. libata transition - call for testing Finally, a major change I have been working on is the transition from the old IDE drivers for PATA controllers to new drivers based on libata. The old IDE subsystem is no longer developed and some drivers do not properly support all the hardware that their libata-based counterparts do. However, while the IDE drivers generate device names beginning with 'hd', libata presents PATA devices as SCSI devices and generates device names beginning with 'sd'. In a system that already has other SCSI or SCSI-like disks, names may change somewhat unpredictably. Similar problems exist for PATA CD/DVD and tape devices. So, while the transition doesn't involve any kernel hacking, it does require a complex upgrade process. Any configuration files that refer to IDE or SCSI device names may need to be changed. It is not enough to switch to only SCSI device names, since we cannot know what they are in advance and the configuration files should continue to work with kernel versions from before and after the transition. In the case of disks we can normally use the partition label or UUID: many tools understand the LABEL= and UUID= syntax, and for others we can refer to the symlinks under /dev/disk created by udev. In the case of CD/DVD devices, we can use the aliases /dev/cdrom etc. created by udev. In the case of tape devices, however, you're on your own. In experimental, kernel image packages depend on a new package linux-base which implements this transition (from 2.6.33-1~experimental.2; the previous version was broken). The postinst script will prompt you to make changes automatically or manually. It can convert most bootloader configuration files, /etc/fstab, the udev CD aliases configuration and the initramfs-tools resume partition. It can also label partitions that don't have a label or UUID. Please do test it and verify that its changes are correct. All changed configuration files are backed up with a suffix of '.old' (or '^old' in one case).

Producing udebs How to patch That s actually quite trivial. Most packages from Josselin s 1st and 2nd lists are libraries, and generating udebs for libraries is easy. Basically it boils down to declaring a new binary package:

• Duplicate the libfooN entry in debian/control.
• Add the -udeb suffix to the Package line.
• Add a Package-Type: udeb line.
• Make sure Section is debian-installer.
• (Maybe) make sure Priority is only optional.

Now, populating it might depend on the considered source package, but thankfully, pkg-x11 packages are quite sane, using debhelper s dh_install. So duplicating debian/libfooN.install into debian/libfooN-udeb.install is usually sufficient. It s a bit trickier when dh_movefiles is used, but calling install and cp -r after the dh_movefiles call should be sufficient. Next step, tweak shlibs for libfooN. Without that step, a libbarM package built against libfooN would get libfooN in Depends, which is not appropriate. The idea is to get libfooN-udeb instead. Trivial: add --add-udeb=libfooN-udeb to the dh_makeshlibs call. Possible issues at that step:

• debian/shlibs or debian/libfooN.shlibs is used instead of dh_makeshlibs. Trivial fix: feed its contents to dh_makeshlibs through its -V argument, and delete that file. Caveat: when it comes to applying a patch removing that file, make sure it s actually removed, and not just empty. I m going to file a bug about that, the default behaviour with empty files seems a bit buggy.
• Several libraries are shipped: one can t pass several --add-udeb arguments. Solution: use several dh_makeshlibs -p$pkg -V$depends --add-udeb=$pkg-udeb call. (I thought it was needed for libx11 but I can t read, actually: libxcb1 libx11-xcb1.)

Various checks to perform:

• Make sure debian/libfooN/DEBIAN/shlibs is correct: it should have a udeb line. Example:

  libfoo N libfooN (>= some-version)
  udeb: libfoo N libfooN (>= some-version)
  

• Make sure the generated udebs have no non-udeb packages mentioned in Depends. Since I m still at the proof-of-concept stage, I ve iterated when I encountered such additional packages, without checking what was actually needed. The idea is to get rid of extra stuff once the final needs figured out. Of course, one has to track down the dependencies to get the packages incrementally built and installed (for the updated shlibs files).

Results (These files might be moved around during the following steps, but this post shall be kept up-to-date and point to them at all time.) Legend:

• Red box: package needing a patch.
• Green box: package already udeb-aware.
• Label with [n]: package patched at step N.
• Label with a *: warning when patching, shlibs file to be removed.

Graphs are scaled down to 75%, click for full size. Step 1:

• Patches: udebs-1/diffs/
• Udebs: udebs-1/udebs/
• Notes: as explained above, libx11-xcb1 is probably not needed.
• Dependency graph:
  

Step 2:

• Patches: udebs-2/diffs/
• Udebs: udebs-2/udebs/
• Dependency graph:
  

Adding udebs to a d-i image Basic d-i image General guidelines (see d-i documentation for detailed installation, needed packages, etc.):

svn co svn://svn.debian.org/svn/d-i/trunk/ debian-installer-trunk
cd debian-installer-trunk/installer/build
make build_netboot

If that breaks (e.g. due to missing packages), make sure there s a similar entry in sources.list.udeb:

deb http://yourmirror/debian squeeze main/debian-installer

If everything goes well, the generated image should be available as dest/netboot/mini.iso, and tmp/netboot/udeb.list should contain the list of embedded udebs. To give it a try:

qemu-system-x86_64 -cdrom dest/netboot/mini.iso

Actually adding udebs Drop all additional udebs under localudebs/. Needed Packages and Packages.gz should be generated on the fly (but beware, secure apt is then disabled, even for non-local udebs). Now, tell d-i we want some more packages. If all of them are wanted, one can do that:

for i in localudebs/*.udeb; do \
  dpkg-deb --showformat='$ Package \n' -W $i; \
done > pkg-lists/local

(Update: Thanks to Guillem for pointing out a better and cleaner way than dpkg --info $i grep Package: awk ' print $2 ') Generate a new image (note the rebuild part of the target):

make rebuild_netboot

Make sure they got added by looking in tmp/netboot/udeb.list and profit! What s next Future steps include:

• Making sure those udebs are actually usable.
• Reworking Pango/Cairo/Gtk udebs.
• Taking a look into X itself.

Stay tuned.

The MirBSD Midi-ISO (bi-arch manifold boot) and NetInstall for both i386 and sparc have been upgraded to the 20091226 snapshot (sorry for the delay). A separate news announcement will be done when a full ISO (MirBSD + MirGRML) is done. Other than that, I have fixed a couple of things all over the place, jupp for example. The planned release of mksh R39b is still not done though, as I m only human as well, and too much hacking isn t something one can do without relaxing some in between. On the Debian front, my RCBD #1 was continued, here s #1 results:

• Carry over from day #1
• bug #552791 acorn-fdisk Copyright file does not contain verbatim copy of the license or a pointer to one
• bug #562647 gidentd Does not work with ipv4 after recent change in netbase
• bug #558812 dietlibc incorrect license in debian/copyright
• bug #531937 autossh FTBFS on mipsel due to missing -fPIC
• New ones
• bug #563522 acorn-fdisk cleanup patches
• bug #563525 gidentd cleanup patches

Explanations: I did go overboard during the first patching session, but I suppose this is what the NM learning period is for too. The autossh maintainer said thanks and will probably integrate my patches, so I don t need to NMU. I could close the dietlibc bug. The other two didn t look as good, I had to separate the fix for the RC bug (and other required fixes, such as ftp-master rejects there were none though) and my other fixes; I submit the former as NMU diffs again and pointed Zack to the .dsc files, and opened the aforementioned two new bugs with the rest of the diffs, so the proper maintainers can take and apply them. There s questioning if gidentd should be removed (see the PR for more); the acorn-fdisk upstream (arm-fdisk it s called there) is not actively developing but will receive patches; the autossh maintainer said thanks but I didn t yet hear back from upstream. The binutils as intel_mode bug was fixed upstream and in experimental for my case, but I had to reopen things because the variant documented in binutils-current still doesn t work, so others (who use the new, more intel-like, syntax) don t run into it. Luk sent me a request to do more mipsel-FTBFS-due-to-toolchain-bugs workarounds. Will do (but can t promise to do so before the upcoming BSP. Robert Millan incorporated something like manifold-boot into GRUB 2, after I described it to him (the debian-bsd@ people are currently sorting out some heisenbugs with it, though). Now there s three variants (but then, this helps spotting bugs that don t appear in all implementations). sendmail 8.13.4 is out, I wonder when OgreBSD will upgrade I could do it myself again, but this time it s not that urgent. Still waiting for the TLS extension, though

xm list grep <VMName>
xm block-list <VMName_Number>
xm block-detach <VMName_Number> <VMName_Vdev> -f
xm block-attach <VMName_Number> file:///iso/disk.iso \
hdc:cdrom r

RCBW - week #4 RC bugs squashed this week by yours truly:

• 21/09/2009 bug #527785 - apertium-es-ro - XML validation errors and apertium 3.1 build
• 22/09/2009 bug #527797 - apertium-oc-ca - XML validation errors and apertium 3.1 build
• 23/09/2009 bug #521900 - apt-dpkg-ref - more specific Lisp build: clisp cmucl (patch from Peter De Wachter)
• 24/09/2009 bug #527791 - apertium-oc-es - XML validation errors and apertium 3.1 build
• 25/09/2009 bug #505815 - cdrbq - force a (non OR-ed) dependency on wish
• 26/09/2009 bug #474146 - arch2darcs - FTBFS no longer reproducible (downgrade severity)
• 27/09/2009 bug #521920 - etherboot - detect GNU mtools version (path by Daniel Schepler) and build against grub-legacy

You're welcome to join the initiative!, in doing so please tag the blog post as "debian" and "rc".

Live-initramfs is a fork of Ubuntu s casper for use within Debian. Nowadays several Debian based live systems are using live-initramfs to build an initramfs suited to boot live systems. Besides debian-live and all its users that s at least Grml (and its derivates) and FAI. I m maintaining live-initramfs for Grml so I know the limitations of live-initramfs. Yes, it has some design flaws but upstream finally decided to take the time to redesign it. That s why I m posting this: I would like to see even more Debian based systems adopting live-initramfs (hello Sidux!). This would improve compatibility with regards to similar bootoptions and features as well as better cooperation among the teams. Of course this will be possible only if live-initramfs is capable of handling all the necessary customization and configuration tasks that different live systems require nowadays. So this are my current considerations regarding the redesign of live-initramfs:

• support all currently available bootoptions (for a clean upgrade path and wide adoption of live-initramfs 2.x)
• provide a possibility to use the rootfs from exactly the device that the kernel/initramfs is booting from (this is especially important if users have several similar systems on their usb drives)
• scan removable devices always before any harddisk devices by default
• provide a bootoption which displays the currently executed code:
  • avoids panic on user s side if something takes longer than usual, so let s inform the user instead
  • debugging is essential if anything fails, though:
  • rebuilding the initramfs with set -x isn t a solution for normal users
• bootoption toram should run some free memory checks
• support custom hooks via something like hooks/live-custom (so other distributions don t have to patch live-initramfs for adding additional stuff)
• consider inclusion of Grml s patches (at least what you think might be interesting for you)
• split scripts/[live-bottom] into a separate package:
  • provide just the core files for booting in live-initramfs package, move debian-live specific stuff to a separate package
  • but please don t just drop 23networking for PXE booting
• write messages through a clean interface:
  • so no /dev/console workarounds are necessary to get around the file descriptor hacks
  • support keeping the system completely quiet (for example for bootsplashes)
  • support custom failure messages (debian-live won t have much fun if they receive distribution specific bugreports)
  • allow customization of the messages so every system can use its own look and feel for the boot messages
• provide status bars wherever possible:
  • when searching for the squashfs file this can take a while (quoting a Grml user: cdrom -> ie -> activex -> fast ethernet -> OOB -> IDE -> Grml ), so optionally display which device is scanned
  • toram is annoying without any status information (displaying progress is possible e.g. using rsync)

If you re interested in adopting live-initramfs in your distribution/system feel free to contact me (mika [at] debian.org). If you do have any further wishes for the redesign please let me know so I can forward them to upstream.